{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Diffusion is a random process that is computationally demanding. You may need to run the [DiffusionPipeline](https://huggingface.co/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline) several times before getting a desired output. That's why it's important to carefully balance generation speed and memory usage in order to iterate faster,\n",
    "\n",
    "This guide recommends some basic performance tips for using the [DiffusionPipeline](https://huggingface.co/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline). Refer to the Inference Optimization section docs such as [Accelerate inference](https://huggingface.co/docs/diffusers/main/en/./optimization/fp16) or [Reduce memory usage](https://huggingface.co/docs/diffusers/main/en/./optimization/memory) for more detailed performance guides."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Memory usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reducing the amount of memory used indirectly speeds up generation and can help a model fit on device.\n",
    "\n",
    "The [enable_model_cpu_offload()](https://huggingface.co/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline.enable_model_cpu_offload) method moves a model to the CPU when it is not in use to save GPU memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from diffusers import DiffusionPipeline\n",
    "\n",
    "pipeline = DiffusionPipeline.from_pretrained(\n",
    "  \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
    "  torch_dtype=torch.bfloat16,\n",
    "  device_map=\"cuda\"\n",
    ")\n",
    "pipeline.enable_model_cpu_offload()\n",
    "\n",
    "prompt = \"\"\"\n",
    "cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California\n",
    "highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain\n",
    "\"\"\"\n",
    "pipeline(prompt).images[0]\n",
    "print(f\"Max memory reserved: {torch.cuda.max_memory_allocated() / 1024**3:.2f} GB\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inference speed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Denoising is the most computationally demanding process during diffusion. Methods that optimizes this process accelerates inference speed. Try the following methods for a speed up.\n",
    "\n",
    "- Add `device_map=\"cuda\"` to place the pipeline on a GPU. Placing a model on an accelerator, like a GPU, increases speed because it performs computations in parallel.\n",
    "- Set `torch_dtype=torch.bfloat16` to execute the pipeline in half-precision. Reducing the data type precision increases speed because it takes less time to perform computations in a lower precision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import time\n",
    "from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler\n",
    "\n",
    "pipeline = DiffusionPipeline.from_pretrained(\n",
    "  \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
    "  torch_dtype=torch.bfloat16,\n",
    "  device_map=\"cuda\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Use a faster scheduler, such as [DPMSolverMultistepScheduler](https://huggingface.co/docs/diffusers/main/en/api/schedulers/multistep_dpm_solver#diffusers.DPMSolverMultistepScheduler), which only requires ~20-25 steps.\n",
    "- Set `num_inference_steps` to a lower value. Reducing the number of inference steps reduces the overall number of computations. However, this can result in lower generation quality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config)\n",
    "\n",
    "prompt = \"\"\"\n",
    "cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California\n",
    "highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain\n",
    "\"\"\"\n",
    "\n",
    "start_time = time.perf_counter()\n",
    "image = pipeline(prompt).images[0]\n",
    "end_time = time.perf_counter()\n",
    "\n",
    "print(f\"Image generation took {end_time - start_time:.3f} seconds\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generation quality"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many modern diffusion models deliver high-quality images out-of-the-box. However, you can still improve generation quality by trying the following.\n",
    "\n",
    "- Try a more detailed and descriptive prompt. Include details such as the image medium, subject, style, and aesthetic. A negative prompt may also help by guiding a model away from undesirable features by using words like low quality or blurry.\n",
    "\n",
    "    ```py\n",
    "    import torch\n",
    "    from diffusers import DiffusionPipeline\n",
    "\n",
    "    pipeline = DiffusionPipeline.from_pretrained(\n",
    "        \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
    "        torch_dtype=torch.bfloat16,\n",
    "        device_map=\"cuda\"\n",
    "    )\n",
    "\n",
    "    prompt = \"\"\"\n",
    "    cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California\n",
    "    highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain\n",
    "    \"\"\"\n",
    "    negative_prompt = \"low quality, blurry, ugly, poor details\"\n",
    "    pipeline(prompt, negative_prompt=negative_prompt).images[0]\n",
    "    ```\n",
    "\n",
    "    For more details about creating better prompts, take a look at the [Prompt techniques](https://huggingface.co/docs/diffusers/main/en/./using-diffusers/weighted_prompts) doc.\n",
    "\n",
    "- Try a different scheduler, like [HeunDiscreteScheduler](https://huggingface.co/docs/diffusers/main/en/api/schedulers/heun#diffusers.HeunDiscreteScheduler) or [LMSDiscreteScheduler](https://huggingface.co/docs/diffusers/main/en/api/schedulers/lms_discrete#diffusers.LMSDiscreteScheduler), that gives up generation speed for quality.\n",
    "\n",
    "    ```py\n",
    "    import torch\n",
    "    from diffusers import DiffusionPipeline, HeunDiscreteScheduler\n",
    "\n",
    "    pipeline = DiffusionPipeline.from_pretrained(\n",
    "        \"stabilityai/stable-diffusion-xl-base-1.0\",\n",
    "        torch_dtype=torch.bfloat16,\n",
    "        device_map=\"cuda\"\n",
    "    )\n",
    "    pipeline.scheduler = HeunDiscreteScheduler.from_config(pipeline.scheduler.config)\n",
    "\n",
    "    prompt = \"\"\"\n",
    "    cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California\n",
    "    highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain\n",
    "    \"\"\"\n",
    "    negative_prompt = \"low quality, blurry, ugly, poor details\"\n",
    "    pipeline(prompt, negative_prompt=negative_prompt).images[0]\n",
    "    ```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Diffusers offers more advanced and powerful optimizations such as [group-offloading](https://huggingface.co/docs/diffusers/main/en/./optimization/memory#group-offloading) and [regional compilation](https://huggingface.co/docs/diffusers/main/en/./optimization/fp16#regional-compilation). To learn more about how to maximize performance, take a look at the Inference Optimization section."
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 4
}
